Picture transmission method, picture transmission method program, storage medium which stores picture transmission method program, and picture transmission apparatus

ABSTRACT

A picture transmission method is disclosed which involves encoding moving picture data and transmitting the encoded data. The picture transmission method includes the steps of encoding firstly the moving picture data into a moving picture stream for transmission at a predetermined data transfer rate, and encoding secondly, based on position information acquired from a transmission destination, a part of the moving picture data which constitutes a partial picture region for transmission at a higher resolution than the first encoding step.

CROSS REFERENCE

This application is a continuation of and is based upon and claims thebenefit of priority under 35 U.S.C. §120 for U.S. Ser. No. 10/133,420,filed Apr. 29, 2002, the entire contents of this application isincorporated herein by reference. U.S. Ser. No. 10/133,420 claims thebenefit of priority under 35 U.S.C. §119 from Japanese PatentApplication No. 2001-133770, filed respectively on May 1, 2001.

BACKGROUND OF THE INVENTION

The present invention relates to a picture transmission method, apicture transmission method program, a storage medium which stores thepicture transmission method program, and a picture transmissionapparatus. More particularly, the invention relates to techniques fordistributing a moving picture stream over a network. According to theinvention, picture data as a whole are first transmitted when compressedat a high compression rate. Based on position information acquired froma given transmission destination, part of the picture data representinga partial picture region is transmitted at a higher resolution than theremaining regions. That is, a user-designated partial region of eachpicture is compressed at a reduced data compression rate for a movingpicture streaming service, so that users may watch desired portions ofthe streaming moving pictures reproduced at a higher resolution.

Conventionally, moving picture streams are distributed over a networksuch as the Internet using a real-time compression and transmissionsystem such as MPEG4 (Moving Picture Experts Group Phase 4).

MPEG4 represents techniques for efficiently compressing moving picturedata through the effective use of correlations between continuouspictures. Where moving picture data streams are distributed using MPEG4,the data compression rate is varied depending on the current amount ofencoded data so that the data may be transmitted at a bit ratecommensurate with the capacity of communication lines being used.

With such a real-time compression and transmission system in use, higherdata compression rates necessarily increase the amount of the datadropped from the original moving picture data and a transmissiondestination may not be able to reproduce clear pictures (i.e. picturesof high qualities). On the other hand, reduced data compression rateseventually make it impossible to transmit moving picture data overcommunication lines that have limited capacities.

One proposed solution (ROI) to the above problem involves reducing thedata compression rate for only a specific region of each moving picture.In other words, only that part of each picture which users arepresumably desirous of watching at higher quality is transmitted athigher resolution while the remaining portions which users do not needin detailed picture, such as background, are left to have lower quality.The conventional solution is supposed efficiently to transmit movingpicture data over communication lines subject to limited capacities.

One disadvantage of the solution above is that the specific region ofeach picture selected for the high-resolution transmission may differfrom the one that is desired by a user. Different users have differentpreferences. In a football game broadcast, for example, some users maylook for a detailed picture of a particular player on defense side whileother users may prefer watching a high-resolution image of anotherplayer on offence side.

Transmitting picture portions that are not desired by a user at highresolution may be wasteful. Under the conventional scheme, a number ofusers may miss high-quality images of their preference.

As outlined above, if only a fixed region of each picture is compressedat a reduced data compression rate for higher-quality distribution overthe network by a moving picture streaming service, some users aresatisfied but others are not. The conventional method has failed toaddress the diverse user preferences where pictures are transmitted on apartial-region enhanced-quality basis.

The present invention has been made in view of the above circumstancesand provides a picture transmission method, a picture transmissionmethod program, a storage medium which stores the picture transmissionmethod program, and a picture transmission apparatus for adequatelyaddressing user preferences so that a user-designated specific region ofeach picture is compressed at a lower data compression rate forhigher-resolution distribution through a moving picture streamingservice.

SUMMARY OF THE INVENTION

According to the first aspect of the present invention, there isprovided a picture transmission method for encoding moving picture dataand transmitting the encoded data, the method comprising the steps of:

encoding firstly said moving picture data into a moving picture streamfor transmission at a predetermined data transfer rate; and

encoding secondly, based on position information acquired from atransmission destination, a part of said moving picture data whichconstitutes a partial picture region for transmission at a higherresolution than said first encoding step.

According to the second aspect of the present invention, there isprovided a picture transmission method program for encoding movingpicture data and transmitting the encoded data, the program comprisingthe steps of:

encoding firstly said moving picture data into a moving picture streamfor transmission at a predetermined data transfer rate; and

encoding secondly, based on position information acquired from atransmission destination, a part of said moving picture data whichconstitutes a partial picture region for transmission at a higherresolution than said first encoding step.

According to the third aspect of the present invention, there isprovided a storage medium which stores a picture transmission methodprogram for encoding moving picture data and transmitting the encodeddata, the program comprising the steps of:

encoding firstly said moving picture data into a moving picture streamfor transmission at a predetermined data transfer rate; and

encoding secondly, based on position information acquired from atransmission destination, a part of said moving picture data whichconstitutes a partial picture region for transmission at a higherresolution than said first encoding step.

According to the fourth aspect of the present invention, there isprovided a picture transmission apparatus for encoding moving picturedata and transmitting the encoded data, the apparatus comprising:

first encoding means for encoding said moving picture data into a movingpicture stream for transmission at a predetermined data transfer rate;and

second encoding means which, based on position information acquired froma transmission destination, encodes a part of said moving picture datawhich constitutes a partial picture region for transmission at a higherresolution than said first encoding means.

Other objects, features and advantages of the invention will become moreapparent upon a reading of the following description and appendeddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a moving picture distribution systempracticed as a first embodiment of the present invention;

FIG. 2 is a plan view showing relations between the overall view of apicture and a specific region of that picture having a higher resolutionthan the rest of the picture as displayed by the system of FIG. 1;

FIG. 3 is a flowchart of steps performed by the transmitting side in themoving picture distribution system of FIG. 1; and

FIG. 4 is a block diagram of a moving picture distribution systempracticed as a second embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of this invention will now be described in detailwith reference to the accompanying drawings.

(1) First Embodiment (1-1) Structure of the First Embodiment

FIG. 1 is a block diagram of a moving picture distribution system 1practiced as the first embodiment of this invention. In the movingpicture distribution system 1, a transmitting side 3 transmits a movingpicture stream over a network 2 such as the Internet for reception by areceiving side 4.

On the transmitting side 3, a camera 5 takes pictures of a desiredobject and outputs video data DV constituting moving pictures. A framememory 6 holds the video data DV temporarily before output.

A region extraction device 7 forwards the video data DV unmodified fromthe frame memory 6 to a general view encoding device 8. The regionextraction device 7 may then be notified of position information DP, asshown in FIG. 2, from a network reception device 13 regarding the videodata DV destined for the general view encoding device 8. In that case,the region extraction device 7 establishes a partial picture region AR1based on the position information DP and outputs video data DV1constituting the partial region AR1 to a high-resolution pictureencoding device 10.

The position information DP is information which denotes the displayposition selected by a user on the receiving side 4 within the generalview constituted by the video data DV. When thus designated, theposition information DP is transmitted from the receiving side 4 inresponse to the user's operation. Given the position information DP, theregion extraction device 7 establishes the partial region AR1 extendingin the horizontal and perpendicular directions from a positiondesignated by the position information DP as a center point of a pictureportion, before outputting the video data DV1 constituting the partialregion AR1 to the high-resolution picture encoding device 10.

When notified of the position information DP from the network receptiondevice 13, the region extraction device 7 keeps outputting to thehigh-resolution picture encoding device 10 the video data DV1 about thepartial region AR1 for a predetermined period of time measured by aninternal timer. When the predetermined time period has elapsed, theregion extraction device 7 stops sending the video data DV1 to thehigh-resolution picture encoding device 10.

Based illustratively on MPEG4, the general view encoding device 8compresses the video data DV from the region extraction device 7 at arelatively high data compression rate prior to output. The output datafrom the general view encoding device 8 are placed by a networktransmission device 11 onto the network 2. In this manner, thetransmitting side 3 distributes all video data DV obtained from thecamera 5 over the network 2 after having the data compressed at arelatively high data compression rate.

Using the same predictive coding scheme as the general view encodingdevice 8, the high-resolution picture encoding device 10 compresses thepartial region video data DV1 from the region extraction device 7 priorto output. During the compression process, the high-resolution pictureencoding device 10 compresses the video data DV1 at a lower datacompression rate than the general view encoding device 8 for output.That is, the high-resolution picture encoding device 10 outputs encodedmoving picture data at a higher resolution than the general viewencoding device 8 outputting its moving picture stream.

A network transmission device 12 places the output data from thehigh-resolution picture encoding device 10 onto the network 2. Based onthe position information DP, the transmitting side 3 transmits theselected part of the video data DV from the camera 5 at a higherresolution than the rest of the video data DV.

The network reception device 13 acquires the position information DPfrom the receiving side 4 and forwards the acquired information to theregion extraction device 7. Accordingly, the region extraction device 7in the moving picture distribution system 1 transmits the general viewvideo data DV at the higher data compression rate while feeding thevideo data DV1 representing the partial picture region designated by thereceiving side 4 at the higher resolution.

On the receiving side 4, a network reception device 15 receives themoving picture stream from the network transmission device 11 andforwards what is received to downstream devices. A general view decodingdevice 16 decodes the video data DV out of the moving picture streamcoming from the network reception device 15, before outputting thedecoded video data DV.

Another network reception device 17 on the receiving side 4 receives themoving picture stream coming from the network transmission device 12 andforwards what is received to downstream devices. A high-resolutionpicture decoding device 18 decodes the video data DV1 out of the movingpicture stream coming from the network reception device 17, beforeoutputting the decoded video data DV1.

A picture composition device 19 substitutes the video data DV1 from thehigh-resolution picture decoding device 18 for part of the video data DVfrom the general view decoding device 16 before outputting the compositevideo data to a frame memory 20. The frame memory 20 temporarily retainsthe output data from the picture composition device 19 before outputtingthe data to a display device 21. The display device 21 displays the datafrom the frame memory 20.

An input device 22 is composed of two-dimensional coordinate inputtingmeans such as a mouse. The input device 22, when operated by the user,moves a cursor on a display screen of the display device 21. The inputdevice 22, when further operated by the user, outputs coordinates DP ofa specific display position on the display device 21. The outputcoordinate information DP is sent by a network transmission device 24over the network 2 to the transmitting side 3.

(1-2) How the First Embodiment Works

In the moving picture distribution system 1 of the above constitution(FIG. 1), the video data DV acquired by the camera 5 are input to thegeneral view encoding device 8 for data compression into the movingpicture stream. After the compression, the moving picture stream istransmitted over the network 2 to the receiving side 4. On the receivingside 4, the general view decoding device 16 decodes the original videodata DV out of the received stream. Given the decoded video data DV, thedisplay device 21 displays moving pictures reflecting the data.

The moving picture distribution system 1 thus allows moving picturestaken by the camera 5 to be viewed on the display device 21. Because themoving picture stream thus transmitted has been compressed by thegeneral view encoding device 8 at a high data compression rate, thestream lacks detailed information and has the correspondingly lowerpicture quality than the original video data DV when displayed.

If a user wants a specific part of the general view to be displayed inmore detail, the user operates the input device 22 to designate thatpart on the screen of the display device 21. In turn, the positioninformation DP about the designated part is reported to the transmittingside 3 (FIG. 2). The report causes the transmitting side 3 to locate thepartial picture region AR1. The video data DV1 constituting the regionAR1 are output by the region extraction device 7 to the high-resolutionencoding device 10. The encoding device 10 compresses the video data DV1on the user-designated region at a lower data compression rate than thegeneral view video data DV for the moving picture stream. The movingpicture stream from the lower-rate compression is sent to and decoded bythe receiving side 4. The high-resolution moving pictures substitute forpart of the general view constituted by the video data DV.

As described, the first embodiment permits users to view a desiredpartial region of the picture at higher resolution than the remainingregions. The feature is particularly useful in applications such as themoving picture streaming service wherein only a user-designated specificportion of each picture is distributed at the reduced data compressionrate to the user in question. Thus different users can enjoy theirpreferred portions of the moving picture stream at the higher picturequality than the remaining picture portions.

In such a moving picture bit streaming application, it is possible thata target which a user wishes to view in detail may move out of theuser-designated partial region AR1 as time elapses, since the target isconstantly moving in the moving picture. When that eventuality is takeninto account, the position information DP may be reported from thereceiving side 4 together with time frame information. In that case, thetransmitting side 3 transmits the high-resolution moving picture streamduring that time frame and stops sending the stream once that time framehas elapsed. In this manner, the inventive moving picture distributionsystem 1 effectively averts wasteful transmission of the video data thatare no longer desired.

FIG. 3 is a flowchart of steps performed by the transmitting side 3.Following the start of moving picture distribution in step SP1, thetransmitting side 3 goes to step SP2. In step SP2, the transmitting side3 causes the camera 5 to capture moving pictures to be transmitted. Instep SP3, a check is made to see if position information DP is acquiredby the network reception device 13. If the position information DP isjudged acquired, that means a partial picture region needs to beextracted for higher-resolution transmission.

If in step SP3 the position information DP is judged acquired, step SP4is reached in which a timer is set on the region extraction device 7. Instep SP5, the timer starts counting. In step SP6, the positioninformation DP is used as a basis for extracting video data DV1representing a partial picture region AR1. In step SP7, the extractedvideo data DV1 are encoded by the high-resolution picture encodingdevice 10. In step SP8, general view video data DV corresponding to awhole picture are fed to the general view encoding device 8. In stepSP9, the video data DV are encoded. After transmitting two sets ofmoving picture stream to the receiving side 4, the transmitting side 3goes back to step SP2 to process another picture.

If in step SP3 the position information DP is not judged acquired, thetransmitting side 3 goes to step SP10. In step SP10, a check is made tosee if a predetermined period of time has elapsed on the timer startedin step SP5 since the start of a high-resolution moving picture stream.If in step SP10 the predetermined time period is not judged to haveelapsed, then step SP5 is reached. In this case, too, thehigh-resolution moving picture stream is transmitted.

If in step SP10 the predetermined time period is judged to have elapsed,the transmitting side 3 goes to step SP8. In step SP8, only the generalview moving picture stream is output by the general view encoding device8 and the high-resolution moving picture stream is suppressed.

(1-3) Effects of the First Embodiment

The first embodiment of the above-described constitution outputs generalview video data at a high data compression rate while transmitting apartial picture region at a high resolution in accordance with positioninformation acquired from a transmission destination. When a particularportion of each picture is thus designated by individual users fortransmission at the reduced data compression rate (i.e., at highresolution) in the moving picture streaming service, user preferencesare adequately addressed.

(2) SECOND EMBODIMENT

FIG. 4 is a block diagram of a moving picture distribution system 31practiced as the second embodiment of this invention. The moving picturedistribution system 31 distributes moving pictures to a plurality ofdestinations. In the moving picture distribution system 31 of FIG. 4,the component parts with their functionally identical or equivalentcounterparts already indicated in the moving picture distribution system1 of FIG. 1 are designated by like reference numerals, and theirdescriptions are omitted where redundant.

In the moving picture distribution system 31, a transmitting side 33causes a general view encoding device 8 to transmit a moving picturestream commonly to receiving sides 4A and 4B which are transmissiondestinations. More specifically, the transmitting side 33 first directsvideo data DV from a frame memory 6 through a region extraction device37 to the general view encoding device 8. The encoding device 8 encodesthe video data DV into a moving picture stream at a high datacompression rate and transmits the encoded stream to the receiving sides4A and 4B via a network transmission device 11.

The transmitting side 33 allows network reception devices 13A and 13B toacquire position information DPA and DPB respectively from the receivingsides. Given the two sets of position information, the region extractiondevice 37 establishes partial picture regions correspondingly andoutputs video data DV1A and DV1B representing the established pictureregions.

On the transmitting side 33, the video data DV1A and DV1B are encodedrespectively by high-resolution picture encoding devices 10A and 10Bgenerating high-resolution moving picture streams destined for thereceiving sides. The two streams are output to the relevant receivingsides 4A and 4B.

The receiving sides 4A and 4B commonly receive the moving picture streamfrom the general view encoding device 8 and decode the video data DVtherefrom. Furthermore, the receiving sides 4A and 4B receive theindividually transmitted high-resolution moving picture streams anddecode the image data DV1A and DV1B therefrom. When thus decoded, thegeneral view moving picture stream and each of the high-resolutionmoving picture streams are composed by picture composition devices 19Aand 19B. The composed video data are reproduced by display devices 21Aand 21B. Viewing reproduced pictures on the display devices 21A and 21B,users may operate input devices 22A and 22B to send position informationDPA and DPB to the transmitting side 33.

In transmitting moving pictures to a plurality of destinations, thesecond embodiment shown in FIG. 4 sends to each destination the generalview video data at the high data compression rate while feeding partialpicture region data at the reduced data compression rate forhigh-resolution transmission based on the position information acquiredfrom the destination in question. With any particular portion of eachpicture thus designated by users of different destinations fortransmission at the reduced data compression rate in the moving picturestreaming service, user preferences are adequately addressed.

The moving picture stream encoded at the high data compression rate isoutput commonly to a plurality of destinations. In turn, positioninformation is acquired from each of the multiple destinations so that adesired partial region of each picture is designated accordingly foreach destination. Video data representing the partial region are thenencoded at high resolution before being transmitted to the respectivedestinations. This setup requires less transmission line occupancy thanconventional arrangements wherein moving pictures are individuallyprepared and transmitted to a plurality of destinations. Reductions inline occupancy translate into higher levels of efficiency at whichmoving pictures are transmitted.

(3) OTHER EMBODIMENTS

The embodiments above were shown acquiring from a transmissiondestination the center coordinates of a partial picture region desiredby the user in order to establish the picture region to be transmittedat high resolution. However, this is not limitative of the invention. Analternative embodiment of the invention may be arranged directly toaccept the user's command for establishing the partial picture region.

The above embodiments were described as setting the partial pictureregion for high-resolution transmission in a uniquely defined fashionbased on the position information acquired from the transmissiondestination. Alternatively, the position information acquired from thedestination may be subjected to a peripheral feature extraction methodor like process. When the nature or character of the target desired bythe user is determined by such a method, the actual region forhigh-resolution transmission may be established accordingly. Otherdiverse techniques may also be resorted to in establishing the partialpicture region to be transmitted at high resolution.

The above-described embodiments of the invention were shown transmittinga desired partial picture region for a predetermined period of time oncethe region is established for high-resolution transmission in accordancewith the position information acquired from the transmissiondestination. Alternatively, a motion tracking technique or like processmay be used to track the target in motion. This allows the partialpicture region for high-resolution transmission to shift in position orbe enlarged or contracted in area while the target is moving.

The embodiments above were shown stopping the high-resolutiontransmission immediately upon elapse of a predetermined period of time.However, this is not limitative of the invention. An alternativeembodiment of the invention may be arranged to gradually end thehigh-resolution transmission by letting the reduced data compressionrate return progressively to the high data compression rate for thegeneral view transmission. This prevents an abrupt, awkward change ofpicture quality in the moving pictures being displayed on the receivingside.

The above embodiments were described as having the region extractiondevice and the frame memory 6 furnished separately. Alternatively, theframe memory 6 may be incorporated into the region extraction device.

In the above-described embodiments, the general view encoding device andhigh-resolution picture encoding device are furnished separately. In analternative embodiment of the invention, the two encoding devices may bedesigned to share part of their components.

Whereas the embodiments above were shown adopting MPEG4 or like encodingmethods for compressing general view video data for low-resolutiongeneral view transmission while encoding partial picture region data forhigh-resolution moving picture transmission, this is not limitative ofthe invention. Alternatively, many other data compression methods may beadopted for data compression and transmission.

Although the transmitting and receiving sides of each embodiment of theinvention above were described as implemented on a hardware basis, thisis not limitative of the invention. Alternatively, part or all of thetransmitting and receiving sides may be implemented by software.

As described and according to the invention, general view video data aretransmitted at a high data compression rate while a specific region ofeach picture defined as per the position information acquired from atransmission destination is transmitted at high resolution, i.e., at areduced data compression rate. With any particular portion of eachpicture designated by individual users for high-resolution transmissionat the reduced data compression rate in the moving picture streamingservice, user preferences are adequately addressed.

As many apparently different embodiments of this invention may be madewithout departing from the spirit and scope thereof, it is to beunderstood that the invention is not limited to the specific embodimentsthereof except as defined in the appended claims.

1. A picture transmission method for encoding moving picture data andtransmitting the encoded data, the method comprising the steps of:encoding, based on selected position information transmitted from eachof a plurality of user devices, a part of said moving picture data of ahigher resolution than a moving picture stream; and transmitting saidmoving picture data to said plurality of devices while transmitting saidmoving picture data corresponding to the respective selected positioninformation of said encoding step, to each of said plurality of devices,wherein the encoding step includes encoding the established partialpicture regions for transmission at a higher resolution for apredetermined period of time, after which, encoding of the establishedpartial picture regions at the higher resolution is halted after theselected position information has moved out of the partial pictureregion.
 2. A method according to claim 1, further including, when thepredetermined period of time has elapsed, gradually changingtransmission of the data corresponding to the established partialpicture region from the high resolution to a predetermined data transferrate.
 3. A method according to claim 1, wherein the encoding fortransmission at the higher resolution for a predetermined period of timeincludes using a timer and discontinuing the encoding step after thepredetermined period of time has elapsed as determined using the timer.4. A method according to claim 3, further including gradually returningtransmission of data for the established partial picture region from thehigh resolution to a predetermined data transfer rate.
 5. A picturetransmission apparatus for encoding moving picture data and transmittingthe encoded data, the apparatus comprising: an encoding means which,based on selected position information transmitted from each of aplurality of user devices, encodes a part of said moving picture data ofa higher resolution than a moving picture stream, wherein said movingpicture data is transmitted commonly to said plurality of devices whiletransmitting said moving picture data corresponding to the respectiveselected position information to each of said plurality of devices, andwherein the encoding means includes encoding the established partialpicture regions for transmission at a higher resolution for apredetermined period of time, after which, encoding of the establishedpartial picture regions at the higher resolution is halted after theselected position information has moved out of the partial pictureregion.
 6. An apparatus according to claim 5, further including, whenthe predetermined period of time has elapsed, gradually changingtransmission of the data corresponding to the established partialpicture region from the high resolution to a predetermined data transferrate.
 7. An apparatus according to claim 5, wherein the encoding fortransmission at the higher resolution for a predetermined period of timeincludes using a timer and discontinuing the encoding after thepredetermined period of time has elapsed as determined using the timer.8. An apparatus according to claim 7, further including graduallyreturning transmission of data for the established partial pictureregion from the high resolution to a predetermined data transfer rate.9. A picture transmission apparatus for encoding moving picture data andtransmitting the encoded data, the apparatus comprising: an encodingdevice which, based on selected position information transmitted fromeach of a plurality of user devices, encodes a part of said movingpicture data of a higher resolution than a moving picture stream,wherein the moving picture data is transmitted commonly to saidplurality of devices while transmitting said moving picture datacorresponding to the respective encoded selected position information toeach of the plurality of devices, and wherein the encoding device isconfigured to encode the established partial picture regions fortransmission at a higher resolution for a predetermined period of time,after which, encoding of the established partial picture regions at thehigher resolution is halted after the selected position information hasmoved out of the partial picture region.
 10. An apparatus according toclaim 9, further including, when the predetermined period of time haselapsed, gradually changing transmission of the data corresponding tothe established partial picture region from the high resolution to apredetermined data transfer rate.
 11. An apparatus according to claim 9,wherein the encoding for transmission at the higher resolution for apredetermined period of time includes using a timer and discontinuingthe encoding after the predetermined period of time has elapsed asdetermined using the timer.
 12. An apparatus according to claim 11,further including gradually returning transmission of data for theestablished partial picture region from the high resolution to apredetermined data transfer rate.